Michael Zeng, Richard Ryu, Adam Sohn
The exploratory analysis focuses on the sample videos:.
| Train/Test | # Total | # Fake | # Real | # Original |
|---|---|---|---|---|
| Train | 400 | 323 | 77 | 209 |
| Test | 400 | ? | ? | ? |
metadata = pd.read_json('data/train_sample_videos/metadata.json').T
metadata.head()
| label | original | split | |
|---|---|---|---|
| aagfhgtpmv.mp4 | FAKE | vudstovrck.mp4 | train |
| aapnvogymq.mp4 | FAKE | jdubbvfswz.mp4 | train |
| abarnvbtwb.mp4 | REAL | None | train |
| abofeumbvv.mp4 | FAKE | atvmxvwyns.mp4 | train |
| abqwwspghj.mp4 | FAKE | qzimuostzz.mp4 | train |
Here is an example of a real video, together with 2 of the fakes that are based on it:
# this is a real one
play_video('ellavthztb.mp4')
metadata[metadata.original == 'ellavthztb.mp4']
| label | original | split | |
|---|---|---|---|
| bnjcdrfuov.mp4 | FAKE | ellavthztb.mp4 | train |
| dbzpcjntve.mp4 | FAKE | ellavthztb.mp4 | train |
# and this is one of the fakes
play_video('dbzpcjntve.mp4')
Here is a demo (noted this video is a fake):
d = display.display(frames_tracked[0], display_id=True)
i = 1
try:
while i <= len(frames_tracked):
d.update(frames_tracked[i % len(frames_tracked)])
i += 1
except KeyboardInterrupt:
pass
The package also allows us to detect key facials points:
# Visualize with key facial points:
fig, ax = plt.subplots(3, 3, figsize=(18, 12))
for i in range(9):
ax[int(i / 3), i % 3].imshow(view_frames[i])
ax[int(i / 3), i % 3].axis('off')
for box, landmark in zip(view_boxes[i], view_landmarks[i]):
ax[int(i / 3), i % 3].scatter(*np.meshgrid(box[[0, 2]], box[[1, 3]]), s=8)
ax[int(i / 3), i % 3].scatter(landmark[:, 0], landmark[:, 1], s=6)
We adapted and wrote our own pipeline for facial recoginition. It is running at about 15 fps on my Macbook Pro, which is not bad at all. However, here are some challenges we are facing:
play_video('djvutyvaio.mp4')
d = display.display(frames_tracked[0], display_id=True)
i = 1
try:
while i <= len(frames_tracked):
d.update(frames_tracked[i % len(frames_tracked)])
i += 1
except KeyboardInterrupt:
pass